Representing uncertainty on model analysis plots

Model analysis provides a mechanism for representing student learning as measured by standard multiple-choice surveys. The model plot contains information regarding both how likely students in a particular class are to choose the correct answer and how likely they are to choose an answer consistent with a well-documented conceptual model. Unfortunately Bao's original presentation of the model plot did not include a way to represent uncertainty in these measurements. I present details of a method to add error bars to model plots by expanding the work of Sommer and Lindell. I also provide a template for generating model plots with error bars.Comment: Accepted for publication in Phys. Rev. Phys. Edu. Res. Published version will include supplementary file

Identifying and Addressing Specific Student Difficulties in Advanced Thermal Physics

As part of an ongoing multi-university research study on student understanding of concepts in thermal physics at the upper division, I identified several student difficulties with topics related to heat engines (especially the Carnot cycle), as well as difficulties related to the Boltzmann factor. In an effort to address these difficulties, I developed two guided-inquiry worksheet activities (a.k.a. tutorials) for use in advanced undergraduate thermal physics courses. Both tutorials seek to improve student understanding of the utility and physical background of a particular mathematical expression. One tutorial focuses on a derivation of Carnot\u27s theorem regarding the limit on thermodynamic efficiency, starting from the Second Law of Thermodynamics. The other tutorial helps students gain an appreciation for the origin of the Boltzmann factor and when it is applicable; focusing on the physical justification of its mathematical derivation, with emphasis on the connections between probability, multiplicity, entropy, and energy. Student understanding of the use and physical implications of Carnot\u27s theorem and the Boltzmann factor was assessed using written surveys both before and after tutorial instruction within the advanced thermal physics courses at the University of Maine and at other institutions. Classroom tutorial sessions at the University of Maine were videotaped to allow in-depth scrutiny of student successes and failures following tutorial prompts. I also interviewed students on various topics related to the Boltzmann factor to gain a more complete picture of their understanding and inform tutorial revisions. Results from several implementations of my tutorials at the University of Maine indicate that students did not have a robust understanding of these physical principles after lectures alone, and that they gain a better understanding of relevant topics after tutorial instruction; Fisher\u27s exact tests yield statistically significant improvement at the = 0:05 level. Results from other schools indicate that difficulties observed before tutorial instruction in our classes (for both tutorials) are not unique, and that the Boltzmann factor tutorial can be an effective replacement for lecture instruction. Additional research is suggested that would further examine these difficulties and inform instructional strategies to help students overcome them

Student understanding of the Boltzmann factor

We present results of our investigation into student understanding of the physical significance and utility of the Boltzmann factor in several simple models. We identify various justifications, both correct and incorrect, that students use when answering written questions that require application of the Boltzmann factor. Results from written data as well as teaching interviews suggest that many students can neither recognize situations in which the Boltzmann factor is applicable, nor articulate the physical significance of the Boltzmann factor as an expression for multiplicity, a fundamental quantity of statistical mechanics. The specific student difficulties seen in the written data led us to develop a guided-inquiry tutorial activity, centered around the derivation of the Boltzmann factor, for use in undergraduate statistical mechanics courses. We report on the development process of our tutorial, including data from teaching interviews and classroom observations on student discussions about the Boltzmann factor and its derivation during the tutorial development process. This additional information informed modifications that improved students' abilities to complete the tutorial during the allowed class time without sacrificing the effectiveness as we have measured it. These data also show an increase in students' appreciation of the origin and significance of the Boltzmann factor during the student discussions. Our findings provide evidence that working in groups to better understand the physical origins of the canonical probability distribution helps students gain a better understanding of when the Boltzmann factor is applicable and how to use it appropriately in answering relevant questions

A framework for the natures of negativity in introductory physics

Mathematical reasoning skills are a desired outcome of many introductory physics courses, particularly calculus-based physics courses. Positive and negative quantities are ubiquitous in physics, and the sign carries important and varied meanings. Novices can struggle to understand the many roles signed numbers play in physics contexts, and recent evidence shows that unresolved struggle can carry over to subsequent physics courses. The mathematics education research literature documents the cognitive challenge of conceptualizing negative numbers as mathematical objects--both for experts, historically, and for novices as they learn. We contribute to the small but growing body of research in physics contexts that examines student reasoning about signed quantities and reasoning about the use and interpretation of signs in mathematical models. In this paper we present a framework for categorizing various meanings and interpretations of the negative sign in physics contexts, inspired by established work in algebra contexts from the mathematics education research community. Such a framework can support innovation that can catalyze deeper mathematical conceptualizations of signed quantities in the introductory courses and beyond

Identifying Student Difficulties with Entropy, Heat Engines, and the Carnot Cycle

We report on several specific student difficulties regarding the Second Law of Thermodynamics in the context of heat engines within upper-division undergraduates thermal physics courses. Data come from ungraded written surveys, graded homework assignments, and videotaped classroom observations of tutorial activities. Written data show that students in these courses do not clearly articulate the connection between the Carnot cycle and the Second Law after lecture instruction. This result is consistent both within and across student populations. Observation data provide evidence for myriad difficulties related to entropy and heat engines, including students' struggles in reasoning about situations that are physically impossible and failures to differentiate between differential and net changes of state properties of a system. Results herein may be seen as the application of previously documented difficulties in the context of heat engines, but others are novel and emphasize the subtle and complex nature of cyclic processes and heat engines, which are central to the teaching and learning of thermodynamics and its applications. Moreover, the sophistication of these difficulties is indicative of the more advanced thinking required of students at the upper division, whose developing knowledge and understanding give rise to questions and struggles that are inaccessible to novices

Student understanding of Taylor series expansions in statistical mechanics

One goal of physics instruction is to have students learn to make physical meaning of specific mathematical expressions, concepts, and procedures in different physical settings. As part of research investigating student learning in statistical physics, we are developing curriculum materials that guide students through a derivation of the Boltzmann factor using a Taylor series expansion of entropy. Using results from written surveys, classroom observations, and both individual think-aloud and teaching interviews, we present evidence that many students can recognize and interpret series expansions, but they often lack fluency in creating and using a Taylor series appropriately, despite previous exposures in both calculus and physics courses

Identifying Student Difficulties with Heat Engines, Entropy, and the Carnot Cycle

We report on several specific student difficulties regarding the second law of thermodynamics in the context of heat engines within upper-division undergraduate thermal physics courses. Data come from ungraded written surveys, graded homework assignments, and videotaped classroom observations of tutorial activities. Written data show that students in these courses do not clearly articulate the connection between the Carnot cycle and the second law after lecture instruction. This result is consistent both within and across student populations. Observation data provide evidence for myriad difficulties related to entropy and heat engines, including students’ struggles in reasoning about situations that are physically impossible and failures to differentiate between differential and net changes of state properties of a system. Results herein may be seen as the application of previously documented difficulties in the context of heat engines, but others are novel and emphasize the subtle and complex nature of cyclic processes and heat engines, which are central to the teaching and learning of thermodynamics and its applications. Moreover, the sophistication of these difficulties is indicative of the more advanced thinking required of students at the upper division, whose developing knowledge and understanding give rise to questions and struggles that are inaccessible to novices

Quantitatively ranking incorrect responses to multiple-choice questions using item response theory

Research-based assessment instruments (RBAIs) are ubiquitous throughout both physics instruction and physics education research. The vast majority of analyses involving student responses to RBAI questions have focused on whether or not a student selects correct answers and using correctness to measure growth. This approach often undervalues the rich information that may be obtained by examining students' particular choices of incorrect answers. In the present study, we aim to reveal some of this valuable information by quantitatively determining the relative correctness of various incorrect responses. To accomplish this, we propose an assumption that allow us to define relative correctness: students who have a high understanding of Newtonian physics are likely to answer more questions correctly and also more likely to choose better incorrect responses, than students who have a low understanding. Analyses using item response theory align with this assumption, and Bock's nominal response model allows us to uniquely rank each incorrect response. We present results from over 7,000 students' responses to the Force and Motion Conceptual Evaluation

Comparing three methods for teaching Newton’s third law

Although guided-inquiry methods for teaching introductory physics have been individually shown to be more effective at improving conceptual understanding than traditional lecture-style instruction, researchers in physics education have not studied differences among reform-based curricula in much detail. Several researchers have developed University of Washington–style tutorial materials, but the different curricula have not been compared against each other. Our study examines three tutorials designed to improve student understanding of Newton’s third law: the University of Washington’s Tutorials in Introductory Physics (TIP), the University of Maryland’s Activity-Based Tutorials (ABT), and the Open Source Tutorials (OST) also developed at the University of Maryland. Each tutorial was designed with different goals and agendas, and each employs different methods to help students understand the physics. We analyzed pretest and post-test data, including course examinations and data from the Force and Motion Conceptual Evaluation (FMCE). Using both FMCE and course data, we find that students using the OST version of the tutorial perform better than students using either of the other two